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Abstract — The research carried out in this technical paper 
suggests an innovative application of the widely implemented 
eye tracking techniques which can be very helpful to users with 
physical disabilities and enable various mediums of 
communication to the needy. The project mainly comprises of 
three sections namely image capture, image processing and 
cursor control. Adequate illumination needs to be used for 
capturing an image which will provide a well-defined shape of 
the pupil. This is done using infrared light since it is invisible to 
the naked eye and hence will not cause any strain to the user. 
The camera needs to be modified to capture infrared light and 
reject visible light. For this an infrared pass filter is mounted 
right before the image sensor. After obtaining the image, the 
shape of the pupil is recognised using Hough Transform and the 
centre coordinates are extracted. The coordinates provide a near 
accurate point of gaze of the eye and hence helps in determining 
the exact point on the screen where the user is looking. These 
coordinates will instruct the computer mouse to move to a 
specific location. The laptop screen size needs to be calculated 
first to appropriately map the received coordinates of the pupil 
centre. When the pupil cannot be detected for a specific range of 
delay a blink action will be registered and the program will 
instruct the mouse to execute the click function. This application 
of eye tracking can also possibly make way for new 
human-computer interaction methods. 


Index Terms — Eye Tracking, Hough Transform, IR pass 
filters Pupil Detection. 

I. INTRODUCTION 

Eye tracking is a technique of determining either the point 
of gaze or the movement of an eye relative to the person’s 
head. An eye tracker is a device for detecting and recognising 
the different positions and movements of the eye. The cursor 
of a computer basically can access any part and any 
functionality of a computer. Being able to control a cursor 
using only eye movements hence provides access to the 
complete functionality of a computer. MATLAB is a 
powerful tool for image processing because it provides so 
many inbuilt functions in toolboxes like Image processing 
toolbox, Computer system vision toolbox etc. Implementing 
shape detection using Hough transform has been made a lot 
easier by the updated image processing toolbox ver. 8.0. 

II. What is Eye Tracking? 

Eye tracking is a technique for measuring the gaze or 
movement of an eye. This helps identify where the user is 
looking and has various applications. 
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1 . Eye attached tracking 

This type consists of an attachment to the eye such as a special 
contact lens with a magnetic field sensor and the movement of 
this is tracked assuming that the lens will not slip significantly 
as the eye rotates. These lenses can provide extremely 
sensitive and precise recordings of eye movements this can 
greatly help scientists study the patterns of eye movements 
and apply these movements for desired functions. 

2. Optical tracking 

Optical tracking uses a non-contact external method for 
detecting and tracking the eye movement. Generally Infrared 
light is used to obtain corneal reflections which give well 
illuminated images of the eye. These images are sensed by a 
modified video camera or any other modified optical sensor. 
Exact rotation and movements of the eye are tracked with the 
help of the obtained infrared illuminated images. 

Generally the centre of the pupil is determined in these 
corneal reflection images and the coordinates of the centre are 
used for the desired applications. These methods, are widely 
preferred for being very inexpensive and not requiring any 
kind of physical modification to the user’s body. 

3. Electric Potential measurement 

This category involves measuring the electric potentials with 
the help of electrodes placed around the eyes. The eyes can 
manipulate. The derived electric signal is called 
Electroculogram (EOG). When the eyes move from the centre 
towards the periphery, the cornea approaches one electrode 
while the retina approaches the other electrode. This change 
in the electric potential and ultimately the EOG signal is 
measured and the movement of the eye is tracked. 

Before using any eye tracker a simple calibration procedure is 
usually required. Every movement of the eye is tracked and 
specific functions are assigned to the different movement 
patterns and positions of the eye. It is still an emerging field 
with upcoming innovative applications in all kinds of every 
day devices for example in phones, different wearable devices 
etc. 

III. Constructional Details 

The project requires minimal hardware, however some 
modification to the camera is required to obtain well filtered 
infrared images. The camera needs to be mounted on a frame 
which can be worn by the user. The camera used is a regular 
8MP camera with optical focus. 

1. Infrared pass filters- Usually cameras come with an IR 
blocking filter right in front of the image sensor. This 
filter is placed to reject some amount of IR light and pass 
visible light to obtain a good quality image. However for 
this application the reverse of this is required. There 
needs to be an IR pass filter which will block visible light 
and only pass IR light. The commonly used IR blocking 
filters work on the principle of selective blocking. They 
are designed for a cut-off wavelength and the design may 
differ from camera to camera. They block the light with 
wavelength higher than the cut-off wavelength i.e. the IR 
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wavelengths. On the contrary, IR pass filters are designed 
to pass IR wavelengths and block lower wavelengths. 
These filters may be a little difficult to find in the market 
but an IR filter can be found among many of presently 
obsolete items, for example the storage film of the floppy 
or the exposed negative film of reel based cameras. For 
this project a floppy disk film was used. 

2. Modifying the camera-any regular camera consists of two 

main parts, the lens and an image sensor (usually a charge 
coupled device CCD). In the camera used there is a lens 
mount which can be unscrewed. We need to filter the 
visible light and pass infrared light and for this we need a 
filter. However this filter should not affect the resolution 
and the focusing of the camera. Hence this filter is placed 
right before the image sensor. Before placing this filter, 
the IR blocking filter of the camera needs to be removed 
which is an easy process. After the camera has been 
modified it simply needs to be mounted in front of the 
user’s eye. 

3. Mount- In order to create a comfortably wearable mount 

we can use the frame of a pair of any random sunglasses 
and attach the camera on it to detect the eye from a close 
range. 

4. Distance from the screen to the user- This distance needs to 

be optimized appropriately in order to be able to 
accurately differentiate the different positions of the eye 
and properly track the centre. If this distance is too large 
then the movement of the pupil will be very less for 
traversing the range of the screen hence it will become 
difficult to accurately track the centre position. If the 
distance is too less then the eye movement will be too 
large and because of the curvature of the eye 

IV. Formulation 

The first thought that led to the formulation of this project was 
to enlarge the scope of image processing application for the 
benefit of the physically challenged people. Thus, the main 
idea of this project was to enable the physically challenged 
individual to manage the home appliances with ease, which in 
turn would make them equally independent as a normal 
human being. An inspiring example is Professor Stephen 
Hawking, an English theoretical physicist, cosmologist, 
author and Director of Research at the Centre for Theoretical 
Cosmology within the University of Cambridge. Hawking has 
a motor neuron disease related to amyotrophic lateral 
sclerosis (ALS), a condition that has progressed over the 
years. He is almost entirely paralysed and communicates 
through a speech-generating device . Despite his debilitating 
illness, he has done ground-breaking work in physics and 
cosmology, and his several books have helped to make 
science accessible to everyone. 

Our project aims at helping people with similar problems and 
providing them access to a plethora of information and 
possibilities. The project also aims to curb the problem of cost 
of manufacture, as it is evident that no manufacturer will 
initiate any project that would increase the cost of production 
and thus decrease the volume of sales. Also less cost will help 
more number of people benefit from it. 


V. Working 

The project is composed of a webcam, infrared LEDs and a 
Laptop. The purpose of this project is to achieve complete 
functionality of the mouse, just by the movement of the eye. 
This project has been implemented using MATLAB version 
2014a. We use basic principles of image processing like 
colour recognition and shape recognition (circle detection) 
for the detection of an eye pupil. 

The webcam is being mounted on the frame of a sun glass. 
Mounting the camera exactly in front of the eye is crucial as it 
is required to obtain a focused image of the pupil to ensure 
proper detection. The camera captures the live images of the 
eye and sends the data to the computer. Infrared LEDs are 
mounted on either side of the camera lenses. The purpose of 
the infrared LEDs is to provide a well illuminated image of 
the of the eye pupil and at the same time they do not cause any 
strain to the eye. An infrared pass filter has been mounted 
right before the image sensor of the camera. The purpose of 
this filter is to pass infrared light whilst blocking visible light. 
This is essential to obtain a well-defined image of the eye 
pupil as shown in the image below. 


Fig. 1 Greyscale image 

Using colour thresholding we filter black colour components 
from the image to obtain a black and white image. This is 
done because circle detection in a greyscale image may cause 
detection on unwanted circles or may detect the radius or the 
centre inaccurately. For thresholding, a proper threshold 
value needs to be selected to obtain a well filtered binary 
image of the pupil. 



Fig. 2 Binary image with detected circle 


Then using shape detection (circle detection) techniques we 
detect a black circular shape and find out the centre of that 
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circle. The centre coordinates will provide the centre point of 
the gaze of the individual. This will help track the exact point 
where the user is looking. This is necessary for accurate 
tracking of the pupil. Centre detection can be done on the 
greyscale image also but it may not always produce the 
required results. The image below demonstrates circle 
detection in the greyscale image. 


Fig. 3 Greyscale image with detected circle 
The camera repeatedly extracts frames from the video and 
carries out the same colour and shape detection procedure to 
continuously track the centre of the pupil 



Thus we can accurately track the movement of eye pupil using 
MATLAB to assign the co-ordinates of the extreme position 
of the eye. Then MATLAB uses these co-ordinates to instruct 
a computer mouse to move to a specific location on the screen. 
The MATLAB program will continuously track the centre of 
pupil and instruct the mouse to move to corresponding 
co-ordinates. 

A blink of the eye will interrupt the continuous process of 
shape and centre detection of pupil. This interruption will 
execute the clicking action of the computer mouse. A blink 
action is registered when the shape of the pupil cannot be 
detected for a specific delay of time. At this point the program 
instructs the mouse to execute the click action. Thus the user 
achieves complete control of the cursor movement of the 
mouse through eye movement. 

VI. Limitations 

The project finds limitations in the following mentioned 
areas. 

• The project as mentioned earlier is proposed for 
physically handicapped and paralyzed people. It 
requires certain assistance for its operation in the 
areas of setting up the webcam and the laptop as 
well as initiation of the MATLAB program. 


• For the perfect calibration of the webcam with respect 

to the eye it is required to take into consideration the 
movement of the user’s head as well. 

• The process of initiation is a time consuming process 

as it requires accurate set up for proper calibration 
ensuring its smooth functioning. 

• At present, only either the left/right click actions of a 

mouse can be executed. Other actions such as scroll 
and middle button click cannot be executed. This 
will need an extra input either from the other eye or 
some other external input. 

VII. Further applications 

The scope of this project can be widened by increasing the 
number of functions such as managing an onscreen keyboard, 
which would enable the user to type short messages using any 
text editor. Also, programs such as paint can be designed 
specifically for easing the accessibility to the user. 
Applications can be designed to enable control of lights, fans, 
television set and other home appliances. It can also be used 
for operating and thereby accessing various mobile 
functionalities such as an intruder alarm to alert other 
household members of the assistance required or informing 
someone in emergency situations by sending a pre-drafted 
message on selected mobile numbers when the user is 
physically unable to send distress calls. 

VIII. Conclusion 

The project, when implemented by taking proper care of the 
limitations as mentioned enables a physically handicapped or 
paralyzed person to gain complete control over the mouse of a 
laptop or a computer by the movement of the eyes. This 
project enables user to become more independent and makes 
him capable to handle his/her work that can be managed using 
a computer/laptop. 
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